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METHOD AND APPARATUS FOR DETECTING MPLS NETWORK 

FAILURES 

RELATED APPLICATIONS 
5 This application claims priority under 35 U.S.C, § 1 1 9(e) to provisional applications 

serial nos. 60/160,840, filed October 21, 1999, 60/161,277 filed October 25, 1999 and 
60/187,798 filed March 8, 2000, the entire writing and content of which is incorporated by 
reference. 

FIELD OF THE INVENTION 

1 0 This invention relates to data networks. In particular this invention relates to a method 

and apparatus for automatically routing a message upstream through a multi-protocol label 
switching (MPLS) network so as to trigger the re-routing of data onto an alternate path 
through the MPLS network. 
BACKGROUND OF THE INVENTION 

15 Multiprotocol Label Switching (MPLS) is a new technology that combines OSI layer 

2 switching technologies and OSI layer 3 routing technologies. The advantages of MPLS 
over other technologies include the flexible networking fabric that provides increased 
performance and scalability. This includes Intemet traffic engineering aspects that include 
Quality of Service (QoS)/Class of Service (COS) and facilitate the use of Virtual Private 

20 Networks (VPNs). 

The Intemet Engineering Task Force (IETF) defines MPLS as a standards-based 
approach to applying label switching technology to large-scale networks. The IETF is 
defining MPLS in response to numerous interrelated problrais that need immediate attention. 
These problems include, scaling IP networks to meet the growing demands of Intemet traffic, 

25 enabling differentiated levels of IP-based services to be provisioned, merging disparate traffic 
types onto a single IP network, and improving operational efficiency in a competitive 
environment. 

The key concept in MPLS is identifying and marking IP packets with labels and 
forwarding them to a modified switch or router, which then uses the labels to switch the 
30 packets through the network. The labels are created and assigned to IP packets based upon 
the information gathered fi:om existing TP routing protocols. 

The label stack is represented as a sequence of "label stack entries". Each label stack 
entry is represented by 4 octets. 
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0 12 3 
012345678 90123456789012345678901 
+-+.+-+.+-+-+-+.+.+-+-+-+-+-+-4--+-+-+-+-^^ Label 

1 Label I Exp |Sl TTL | Stack 
5 +.+-+-+.+-+.+-+-+-+-+-+-+-+-+-+-+-+-+-+.+-+-+-^^ 

Label: Label Value, 20 bits 
Exp: Experimental Use, 3 bits 
S: BottomofStack, Ibit 
10 TTL: Time to Live, 8 bits 

The label stack entries appear after the data link layer headers, but before any network 
layer headers. The top of the label stack appears earliest in the packet, and the bottom appears 
latest. The network layer packet immediately follows the label stack entry which has the S bit 
15 set. 

Multi-protocol label switching (MPLS) networks are typically comprised of several 
packet-based switching systems interconnected by a variety of media (e.g., coaxial or fiber 
optic cable, unshielded twisted pair or even point-to-point microwave wireless) in a mesh- 
topology network similar to the public switched telephone network. In such a network, there 

20 might be several paths through the network between any two endpoints. MPLS networks 
carry data as packets wherein each packet includes a label on identifying a switched path 
through the network. The data label is appended to data packets so as to define a pathway 
through the network over which the data packets are to be routed. 

A problem with any data network, including an MPLS network, is the amount of time 

25 required to recover fi"om either a link failure or a switch failiire. Empirical data shows that the 
time required to recover firom a network failure can take several seconds to several nmiutes, 
an unacceptably long time. A method and apparatus by which the recovery time for a link or 
switch failure can be reduced to perhaps less than a few hundred milliseconds would be a 
significant improvement over the prior art fault recovery mechanisms used on MPLS 

30 networks to date. A method and apparatus by which a switch over from a working path to a 
protection path would facilitate MPLS network reliability. 
SUMMARY OF THE INVENTION 

In an MPLS data network comprised of various transmission media linking various 
types of switching systems, network fault recovery time is reduced by using a reverse- 
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directed status message that is generated by a data switch that is down-stream from a 
switching system from which data is received. The reverse-directed or upstream status 
message is sent over a pre-determined pathway (i.e. through pre-determined switches and/or 
over pre-determined data links) which originates from a destination switch or node in an 
5 MPLS network to upstream switching systems. This so-called reverse notification tree carries 
a message or messages that are used to indicate the functionality (or non-functionality) of the 
downstream switch, switches or links of the MPLS network. As long as an upstream MPLS 
switching system continues to receive the reverse-directed status message from a downstreani 
switch via the reverse notification tree, the switching systems that receive such a message 

1 0 consider the downstream switch and pathways to be in intact. Accordingly, data packets 
continue to be sent downstream for subsequent routing and/or processing. If the reverse- 
directed status message is lost or discontinued, either because of a switch failure or a link 
failure, the upstream switching system considers the downstream switch or link to have failed 
and thereafter begins executing a procedure by which data is rerouted over an alternate data 

1 5 path through the network. In the preferred embodiment, the altemate data path over which 
downstream information ins sent is a pre-established protection path and is known to a 
protection switch in advance, thereby niinimizing data loss attributable to the time it might 
take to calculate a dynamic altemate protection path. 

Switches in the network and their interconnections can be modeled using a directed 

20 acyclical graph by which a downstream switch knows the identity of the upstream switch to 
which the failure notice should be sent. In the preferred embodiment, at least one upstream 
switch routing the MPLS data re-directs data onto a protection path through the network 
between the same two endpoints by using messages carried over a different pathway, 
discussed hereinafter and referred to as a reverse notification tree. By way of the reverse 

25 notification tree, data loss caused by either a link or switch failure can be minimized by the 
prompt rerouting of the data through an altemate or recovery data path through the network. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a simplified block diagram of an MPLS protection configuration. 
Figure 2 depicts exemplary message flows in an MPLS network. 

30 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

Figure 1 shows a simplified block diagram of a packetized-data.switching network 
1 00. Each of the squares shown in Figure 1 including boxes represented by reference 
numerals 102, 104, 106, 108, 110, 112, 114, 116, 118 and 120 represent one or more types of 
asynchronous switching systems that asynchronously receive data in e.g., packets, cells or 
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frames from an '^lpstream" switch and route, direct, couple or otherwise send the data 
onward to another "downstream" switch logically closer to the ultimate destination for the 
data. By way of example, these switching systems might be internet protocol (IP) routers, 
asynchronous transfer mode (ATM) switches, frame relays switches or other types of 
5 packetized-data switching systems implemented to receive packetized data over a 

transmission line and reroute the data onto one or more output ports to which are connected 
transmission media coupled to other switching systems. 

In Figure 1, switching system number 1, (identified by reference numeral 102) is 
coupled to another switching system, no. 2, (represented by reference numeral 104) and 

10 switching system no. 5 (represented by reference numeral 120) via links L12 and P25 
respectively. Switching system no. 2 and no. 5 are "downstream" from no. 1; no. 1 is 
considered to be "upstream" from switch no. 2 and no. 5, 

Similarly switching system no. 3, (represented by reference numeral 106) is coupled 
to switching systems 2, 4 and 9 (represented by reference numerals 104, 108 and 116 

1 5 respectively) via transmission links L23> L34, and L93 respectively. 

In routing data between switch no. 1 (represented by reference numeral 102) and 
switch no. 7 (represented by reference numeral 1 12) data might be routed between these two 
endpoints through a ''primary " path that is comprised of links that logically or physically 
couple switches 2, 3, 4, 6 and 7 (identified by reference numerals 104, 106, 108, 1 10 and 1 12 

20 respectively). The physical or logical links of the primary path between the endpoints which 
is 1 and 7 are represented by the vectors designated Lu, L23, L34, L46 and Le?. This path is 
known in the art as the working or primary path through the network. The links of the various 
paths shown in Figure 1 (represented by the vectors Ln, L23, L34, L46 and Le?) , and therefore 
the paths themselves, might be constructed of direct pathways (e.g., fiber optic cable, coaxial 

25 cable, unshielded twisted pairs of copper wires, or microwave radio) between the various 
switches. Alternate embodiments of the paths or links between switches of the network of 
Figure 1 would also include using direct pathways, and intermediate switches or switch 
networks, (not shown in Figure 1 , but still part of the path or link coupling one or more 
switching systems to another). By way of example and not of limitation, the data switches 

30 shown in Figure 1 might be IP switches but such ff switches could be linked together using 
one or more ATM switches or ATM networks. 
The MPLS Protection Path 

In an MPLS network, there is almost always a "protection " path, which is an 
alternate path through the network linking two endpoints. The protection patii entry and exit 
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points are usually accessible to only protection switches. A protection switch is a switch that 
can re-route traffic onto a protection pathway. Like the other links described above, a 
protection pathway can be comprised of direct data paths, but also switches or switching 
systems, by which data can be sent through a network between two or more endpoints. 

5 In an MPLS network, a protection path is set up using at least one protection switch 

element so as to be able to carry data from a source to a destination in the event the primary 
path or switch thereof fails for one reason or another. The operation of a protection switch is 
shown in Figure 1 by way of example. 

In Figure 1, a working path between switch 1 and 7 exists through switches 1, 2, 3, 4, 

1 0 6 and 7 and the links between the switches, A protection path for the portion of the working 
path that runs tiirough switches 2, 3, 4 and 6 is the path designated by links P25 and P27 and 
which runs through switch 5, (identified by reference nximeral 120). The protection path 
extends between endpoint switches 1 and 7 but through only switch 5 (identified by reference 
numeral 120). Alternate embodiments of a protection path might extend through multiple 

15 switches. In the network 100, either a link or switch loss between switch 1 and 7 can be 

overcome by re-routing traffic for switch 2 through switch 5 instead. Switch 5 then routes the 
data to switch 7. Switch 1 (identified by reference numeral 102) is considered to be a 
protection switch element. 

Another working path between switch 8 and switch 7 of the network 100 (identified 

20 by reference numerals 1 14 and 1 12 respectively) exists through switches 9, 3, 4, 6 & 7 

(identified by reference numerals 1 16, 106, 108 and 110 respectively) and tiie links between 
them. A protection path for data from switch 8 (reference numeral 1 14) to switch 7 (reference 
numeral 1 12) through the network 100 exists via switch 10, (reference numeral 118) such that 
if data traffic from switch 8 (reference numeral 114) is lost somewhere between switch 9 

25 (reference numeral 116) and switch 7 (reference niraieral 112), switch 9 can re-route such 
data traffic to switch 10 (reference numeral 118). Switch 10 can then route such data to 
switch 7. Switch 9 is therefore considered to be a protection switch element. 

If an alternate data path, i.e. a protection path, is pre-determined, i.e. set up or 
established in advance, data loss attributable to a switch or link failure can be minimized. If a 

3 0 protection switch is pre-programmed to re-route data upon its receipt of an appropriate 

command or signal, the protection switch element can almost immediately start sending data 
to the prop^ destination via the protection path. 
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The Liveness Message 

In the event of a pathway failure causing downstream data to be lost at a downstream 
switch, such as by either a switch failure or a link failure, anywhere along a primary or 
working path, a protection switch element (PSL), such as switch no. 1 (identijBed by 
5 reference numeral 1 02) can re-route data traffic through the protection path so as to have the 
data for the endpoint switch no. 7 delivered as quickly as possible to the endpoint at switch 
no. 7 (identified by reference numeral 1 12). The ability to re-route data to a protection path is 
made considerably more valuable if the decision to switch over to a protection path is based 
upon an affirmative notice that a switch over is needed. In one embodiment, this affirmative 

1 0 notice is in the form of an upstream liveness message, the loss of which indicates a pathway 
failure. As long as a liveness message is received at an upstream switch from a downstream 
switch, the upstream switch can assume that the pathway between the two switches is intact 
and that the downstream switch is functional. 

The liveness message is sent upstream, and is repeated, so long as downstream data is 

1 5 received. In one embodiment, the liveness message is repeated periodically, at some 

appropriate repetition rate, however, an alternate embodiment includes sending an upstream 
message aperiodically, and also without regard to the time between messages. Downstream 
data loss at a switch will trigger fihe switch to inhibit upstream-directed liveness messages. 

In another embodiment, a predetermined downstream liveness message is sent instead 

20 of, or in addition to downstream data. The receipt of a such a downstream liveness message 
will cause a recipient switch to maintain its generation of upstream liveness messages. For 
purposes of claim construction, downstream liveness messages and downstream voice or 
downstream data (considered to be interchangeable for claim construction purposes) are 
considered to be equivalent under the rubric of "a first data message" or a "first message," the 

25 reception of which can either inhibit an upstream liveness message, or in alternate 
embodiments, trigger an upstream Uveness message. 

Those skilled in the art will appreciate that while the loss of a liveness message can 
trigger a protection switch of data to a protection path, an equivalent altemate embodiment of 
the invention includes a method by which a protection switchover is initiated or occurs upon 

30 the receipt of either a liveness message or a notification message to a protection switch 

element In such a case, the liveness message and/or the notification acts as a trigger event to 
cause information to be re-routed onto the protection path. 

In the event of a switch or link failure anywhere between the path endpoint switches 1 
and 7, data re-routing is accomplished faster by using a reverse-directed status message that 
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is sent backward or upstream toward the protection switch element no. 1 (reference numeral 
102) by one or more of the switches 2, 3, 4, 6 or 7 (reference numerals 104, 106, 108, 1 10 or 
1 12) of the primary pathway, links L12, L23, L34> L465 Ley. In the preferred embodiment this 
reverse direction data message is known as a "liveness message" the foraiat of which is a 
5 design choice and dependent upon nature of the switches of the network 100, but the function 
of which is to indicate to upstream switches that the data traffic sent to the downstream 
switch arrived intact and on time. 

The structure of a liveness message will vary depending upon whether the network 
switches are ATM, IP, Ethemet or other types of switches, as those skilled in the art will 

10 recognize. Unlike known fault detection methods, the liveness message is not a copy, or 
loop-back of the downstream data. The salient aspect of the liveness message is that it is an 
informational status message, preferably sent at periodic intervals between adjacent nodes, 
indicating the operational condition of the switch fi-om which it was sent. As set forth above, 
altemate embodiments include sending a Uveness message aperiodically. In at least one of 

15 the foregoing embodiments, the fact that the liveness message is received at an upstream 
switch (with respect to the downstream data) is evidence that the link between the switches, 
over which downstream data would be sent, is intact and that the switch that generated the 
liveness message is at least somewhat functional. 

While the preferred embodiment contemplates that the liveness message is sent 

20 upstream firom a switch, directly to the switch that sent the downstream data, altemate 
embodiments of the invention contemplate that a liveness message, or equivalent thereof, 
could be sent in both upstream and downstream directions, between other nodes, or fi:om one 
switch to an intermediate transfer point, which for pxirposes of claim construction are 
considered to be eqmvalent embodiments. By way of example, with reference to Figure 1 , 

25 switch no. 4 (identified by reference numeral 108) will send a liveness message, upstream to 
switch no. 3 (reference numeral 106) in response to data sent downstream (or perhaps a 
downstream liveness message, sent downstream) from switch no. 3 to switch no. 4. If the 
Uveness message from switch no. 4 is lost by or not received by switch no. 3, switch no. 3 
can infer that either the link R43 between the two switches failed, or switch no. 4 failed. If 

30 switch no. 3 was carrying data from switch no. 9 (reference no. 1 1(5) and which is a 

protection switch element having access to a protection path, switch no. 3 would need to 
inhibit its liveness message to switch no. 9 or gen^ate an error message to switch no. 9, 
thereby instructing switch no. 9 to re-route traffic from switch no. 3, to the protection path 
through switch no. 10 (reference numeral 118) 
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As for data routed through' switch no. 3 that comes from switch no. 2 (reference 
numeral 104), a liveness message loss from switch no, 4 will require switch no. 3 to inhibit 
the Uveness message to switch no. 2, or send an error message to switch no. 2. This procedure 
is then repeated to switch no, 1, instructing switch no. 1 to make a protection switch through 
5 switch no. 5 (reference no. 120). 

When a Uveness message is lost, its failure is considered to be indicative of a path 
failure of either a link or a sv^tch. Still other embodiments of the invention contemplate 
sending a downstream liveness message, sent from an upstream switch to a downstream 
switch thereby indicating to a downstream switch that the upstream switch and link are 
10 functional. 

As set forth above, the format of a liveness message will depend upon the type of 
switching systems used in the network. IP switches and ATM switches will need to comply 
with their respective protocols. Altemative embodiments of the invention would certainly 
contemplate other sorts of Uveness messages having different formats with the saUent feature 

15 of the message being that the message indicates to an upstream switch that downstream 
directed data messages were received by a downstream switch intact. 

In Figure 1, the links over which reverse notification status messages (i.e. the 
upstream liveness messages) are sent, are designated by the reverse directed vectors, one of 
which is shown in Figure 1 (and identified by reference numeral R76). By way of example if 

20 link L67 should fail causing a data loss to the endpoint switch no. 7, the corresponding loss of 
the Uveness message ordinarily sent from switch 7 to switch 6 would provide an indication to 
switch no. 6 that either the link or the switch 7 failed whereupon switch no. 6 would begin 
notifying the protection switch (switch no. 1) upstream from it by way of a reverse 
notification message sent on reverse link that would be sent to switch no. 4, (represented 

25 by reference numeral 108). Similarly, switch no. 4 would thereafter return a reverse 

notification message on reverse link R43 to switch no. 3. Switch no. 3 returns another reverse 
notification message on reverse link R32 to switch 2 which then returns a reverse notification 
message on reverse link R21 to the origination node 1 . 

The ultimate destination of the upstream message, and in this case the reverse 

30 notification message, is a switching node (i.e. a switch or switching system) that is capable of 
re-routing downstream traffic, data or messages onto a diffd-ent path, i.e., a protection path, 
usuaUy comprised of at least a different transmission route, possibly including a different 
transmission media as weU (coax to fiber; fiber to microwave etc.). Whether the upstream 
message goes through another switch on its way to the switching node (which has the 
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capability of re-routing data to the protection path) or is directly sent to the switching node 
from a downstream switch aroimd an intermediate switch (for example, sending a liveness 
message directly from switch 6 to switch 1) would still provide equal functionality in that the 
switching node will eventually receive notification that it needs to re-route traffic, data or 
5 message onto the protection path. Sending the aliveness message directly to the protection 
switch or routing the aliveness message via intervening switches are considered to be 
equivalent embodiments for purposes of claim construction. 

Inasmuch as switch no. 1 in Figure 1 is designated as a ''protection switch element" 
meaning that it is coupled to and capable of routing data onto a protection path P25, the 

10 protection switch element 1 (identified by reference numeral 102) reroutes traffic to switch 
no, 7 via a protection path designated by P25 and P57 and that runs through switch no. 5 
(identified by reference numeral 120). 

In the preferred embodiment, the switches of the network maintain tables of network 
switches upon which incoming data is received and a table of network switches to which 

1 5 outgoing data is routed. By keeping a record of where outgoing data from a switch originates 
from, it is possible for a switch of the network 100 to promptly notify an upstream switch of a 
downstream link or switch failure. 

In the process described above, each of the switches of the network sequentially 
notifies at least one switch upstream from it. Altemate (and for purposes of claim 

20 construction, equivalent) embodiments of the invention could certainly provide upstream 

notification messages directly from any downstream switch to every other upstream switch in 
a pathway. In such an embodiment, switch no, 6 might send a reverse notification message 
directly to the protection switch element 1 via a direct link thereby notifying the protection 
switch to immediately reroute data to the protection path P27 and P57 via switch no. 5. Switch 

25 no. 6 might also send a reverse notification (liveness) message to the other switching systems 
of the network as well. 
The Reverse Notification Tree 

The implementation of the upstream notification message, and its conveyance 
upstream to a protection switch element, is enabled by using an upstream pathway 

30 denominated herein as a reverse notification tree or 'TRNT." The RNT is an *^ipstream" signal 
pathway that allows messages from a protection path end point to be sent ^'upstream" to one 
or more protection path (and working path) starting point switches, nodes or starting points. 
In the preferred embodiment, the RNT passes through the same switches and over or through 
the links that comprise the working path (albeit over different transmission media) and for 
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claim construction purposes can be considered to be "coincident" with the working path. 
Alternate embodiments of the invention would include a reverse notification tree that runs 
through one or more switches or nodes that are not part of the working path, or which are 
only partly "coincident." 
5 With respect to Figure 1, node 7, identified by reference numeral 112, is the RNT 

starting point or head end. Nodes 1 and 9, which are identified by reference nxmierals 102 and 
1 16, are the end points of the RNT and to which upstream protection switch messages would 
be sent from any node or switch between nodes 1, 9 and 7. Intervening nodes 3, 4 and 6, 
identified by reference numerals 106, 108 and 110 respectively, are constituent elements or 

10 parts of the RNT. 

The RNT can be established in association with the working path(s) simply by 
making each switching system along a working path ^'remember" its upstream neighbor (or 
the collection of upstream neighbors whose working paths converge at a network switching 
element and exit as one). A table or other data structure stored in memory (such as RAM, 

15 ROM, EEPROM, or disk) of the switches of the paths can be configured to store data 

identifying switches coupled to a switching system in, or part of a working path as well as a 
protection path. 

With respect to the network shown in Figure 1, Table 1 below shows that incoming or 
"Ingress" RNT messages to switch no. 3 from switch 4 are labeled "N43" (not shown in 

20 Figure 1) and that these messages arrive at switch no, 3 from switch 4 at an inbound or 

"Ingress" interface 134 (not shown in Figure 1). Because switch no. 3 receives downstream 
messages from two (2) different switches, (i.e. switch 2 and switch 9) both of these two 
upstream switches must be sent an upstream notification therefore requiring two separate 
upstream messages from switch 3, Upstream RNT messages to switch 2 are labeled "N32" 

25 and appear or are sent from interface 123. Upstream RNT messages to switch 9 are labeled 
'W3" and are sent from interface 193. 
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Table 1. An inverse cross-connect reverse notification tree 
table for Switch 3 of Figure 1. 

5 The reverse path (iqpstream) to switch 3 from switch 4 is labeled N43; the switch 3 

interface for this data is designated 134. An upstream message received at 134 and that is 
labeled N43, is sent out from switch 3, via the interfaces 123 and 193 and labeled N32 and 
N3 9 respectively. 

Table 2 shows the egress and interface labels of the working or downstream path from 
10 switch 3 and the originating switches for that data. 

The working path (downstream) path from switch 3 is to switch 4 and is labeled 
"L34." The switch 3 interface for this data is designated "134." The data sent downstream 
from switch 3 originates from switch 2 and switch 9, which are referred to in Table 2 as 
"Next Hop" switches. 

1 5 Switch no. 2 originates data to switch no. 3 and that data is received at switch no, 3 on 

interface "12." Data from switch no. 9 is received at switch no. 3 at interface "19." The RNT 
or upstream notification to switch no. 2 leaves switch no. 3 on its RNT interface "123." RNT 
notification to switch no. 9 leaves switch no. 3 from "193." 
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Table 2. An inverse cross-connect table for a hop-by-hop 
reverse notification tree. 

25 

A fault on the link between switch 3 and 4 in the downstream direction can be detected at a 
downstream node, switch 4 perhaps, via either a downstream data loss, a downstream liveness 
message loss or via a path failure (PF) or Imk failure (LF). A path failure (PF) is a software- 
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detected hardware fault. A link failure (LF) is a hardware-detected hardware fault. Upon the 
detection of a fault, the downstream node (switch 4) will periodically transmit fault indication 
signal (FIS) messages to its upstream neighbor (switch 3) (via the uplink R43), which will 
propagate further upstream (using its inverse cross-connect table) until the FIS messages 
5 eventually reach the appropriate Protection Switch Element (switch 1 and/or 9). Receipt of an 
FIS message will trigger the protection switch over of data from the working path to the 
protection path. From Table 1, messages received at switch no. 3 are labeled 'W3." Therefore, 
in Fig. 1, if link L34 has a fault, switch 4 can detect the fault via either a lost downstream 
liveness message from switch no. 3, or by detecting a Link Failure (LF) or Path Failure (PF) and 

10 thereafter start transmitting an FIS packet back to switch 3 on link L23 as represented by a 
message on link R43. From Tables 1 and 2, there are two egress messages and interfaces from 
switch no. 3, which identify the upstream switches that are to be ^'notified" of a failure 
doAvnstream from switch no. 3. (The traffic in the queues of switch 3 will continue to be 
serviced.) By using similar tables, switch 2 in tum will propagate the FIS over the RNT back to 

1 5 SAvitch 1 . The actual protection switch will be performed by switch 1 , after the receipt of the first 
FIS. Switch 3 will stop transmitting FIS messages time units after the transmission of the first 
FIS message. 

In the preferred embodiment, only one RNT is required for all the working paths that 
merge (either physically or virtually) to form the multipoint-to-point "forward" or 

20 "downstream" path. Figure- 1 shows that at least two (2) working paths (one path of which is 
comprised of switch elements 1, 2 and 3 that are identified by reference numerals 102, 104 
and 106; a second path of which is comprised of switch elements 8, 9 and 3 that are identified 
by reference numerals 1 14, 1 16 and 106) converge at switch elraient 3 (identified by 
reference numeral 106). Altemate (and for purposes of claim construction, equivalent) 

25 embodiments would include using multiple RNTs for a single working path that has multiple 
paths that converge at a single node (switches of each path that converges might form 
different RNTs) as well as using multiple RNTs for a single working path. 

The RNT is rooted at an appropriately chosen label switched router ("LSR"), (which 
hereafter is referred to as an MPLS network switch element) along the common segment of 

30 the merged working paths and is terminated at the protection switch elements (PSLs). 

Intermediate network switching elements on the converged working paths typically share the 
same RNT reducing signaling overhead associated with recovery. Unlike schemes that treat 
each label switched path (LSP) independently, and require signaling between a protection 
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switch element and a destination switch individually for each LSP, the RNT allows for only 
one (or a small number of) signaling messages on the shared segments of the LSPs. 

The RNT can be implemented either at Layer 3 or at Layer 2 of the OSI, 7-layer 
protocol stack. In either case, delay along the RNT needs to be carefidly controlled. This may 

5 be accompUshed by giving the highest priority to the fault and repair notification packets, 
which travel along the RNT, We can therefore have a situation where different protection 
domains share a conamon RNT. 

A protection "domain" is considered to be the switches and links of both a working 
path and protection path. For example, in Fig. 1, the protection domain bounded by network 

10 switch element 1 and network switch element 7, is denoted by {1-2-3-4-6-7, 1-5-7}. 

When different protection domains have different RNTs, two cases may arise, 
depending on whether or not any portions of the two domains overlap, that is, have nodes or 
links in common. If the protection domains do not overlap, the protection domains are 
considered to be independent. By virtue of the RNTs in the two domains being diflFerent, 

1 5 neither of the working paths nor the RNTs of the two domains can overlap. In other words, 
failures in one domain do not interact with failures in the other domain. For example, the 
protection domain defined by {9-3-4- 6-7, 9-10-7} is completely independent of the domain 
defined by {11-13-5-15, 11-13-14-15}. As a result, as long as faults occur in independent 
domains, the network shown in Fig. 1 can tolerate multiple faults (for example, simultaneous 

20 failures on the working path in each domain). If protection domains with disjoint RNTs 
overlap, it implies that the protection path of one intersects the working path of the other. 
Therefore, although failures on the working paths of the two domains do not affect one 
another, failures on the protection path of one may affect the working path of the other and 
visa versa. For example, the protection domain defined by {1-2-3-4-6-7, 1-5-7} is not 

25 mdependentofthe domain defined by {11-13-5-15, 11-13-14-15} since LSR 5 lies on the 
protection path in the former domain and on the working path in the latter domain. When 
protection domains have the same RNT, different failures along the working paths may affect 
both paths differentiy. As shown in Fig. 1, for example, working paths 1-2-3-4-5-7 and 9-3-4- 
6-7 share the same RNT. As a result, for a failure on some segments of the working path, 

30 both domains will be affected, resulting in a protection switch in both (for example, the 

segment 3-4-6-7 in Fig. 1). Likewise, for failures on other segments of the working path, only 
one domain may be affected (for example, failxire on segment 2-3 affects only the first 
working path 1-2-3-4-6-7, where as failure on the segment 9-3 affects only the second 
working path 9-3-4-6-7). 



wo 01/029685 PCT/USOO/28992 

There are a number of ways to establish a protection domain, i.e., a working path and 
a protection path through an MPLS network. Establishing a protection path first requires the 
identification of the working path (embodied as some series of switches and path links 
through the MPLS network from a sending node to a destination node). In most cases, the 
5 working path and its corresponding recovery path are specified during a network switch path 
or connection setup procedure, either via a path selection algorithm (running at a centralized 
location or at an ingress network switch element) or via an administrative configuration (e.g. 
a manual specification of switches that comprise the protection path). 

The specification of either a protection or working path, does not, strictly speaking, 
1 0 require the entire path to be explicitly specified. Rather, it reqmres only that the head end 
node or switching node and end or destination switch or node (of the respective paths) be 
specified. In the absence of a destination switch/node specification, the path egress points out 
of the MPLS network or domain need to be specified, with the segments between them being 
" "loosely determined or routed. In other words, a working path would be established 
1 5 between the two nodes at the boundaries of a protection domain via (possibly loose) expUcit 
(or source) routing using LDP/RSVP [label distribution protocol/reservation protocol] 
signaling (alternatively, via constraint-based routing, or using manual configuration), as set 
forth more fiilly below. 

Figure 2 depicts message flows between four (4) different switches of an MPLS 
20 network that employs the path protection techniques disclosed herein. Vertical axes of Figure 
2, identified by reference numerals 202, 204, 206 and 208, represent switching elements 
(shown in Figure 1) of an MPLS network firom which and to which various types of messages 
are received and sent respectively. Switch 202 is upstream firom switches 204, 206 and 208. 
Switch 204 is upstream from switch 206 as switch 206 is upstream from switch 208. 
25 Protection Path Establishment 

A Protection Domain Path is established by the identification of a protection switch or 
node and an end point switch or node in the MPLS network. The protection switch element 
CTSL") initiates the setup of the working LSP and elements and the recovery LSP and 
elements. It is also responsible for storing information about which network switch elements 
30 or portions thereof have protection enabled, and for maintaining a binding between outgoing 
labels specifying the working path and the protection/recovery path. The latter enables the 
. switchover to the recovery path upon the receipt of a protection switch trigger. 

A *label distribution protocol" is a set of procedures by which one LSR (i.e., a 
network switch element) informs anoflier of the label bindings it has made. *TLabel binding" 
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is a process by which a message to be sent from a soxjrce to a destination is associated with 
various labels between the nodes that he along the way, between the source and destination. 
By way of example, in Figure 1, a message to be sent from switch 1 to switch 7 is associated 
or boimd to travel to switch 7 through switch 2 by, or using, the label L12 that is first 
5 associated with the message at, or by, switch 1 . Switch 2 in turn associates messages labeled 
L12 as bound for switch 3 and re-labels them as L23, Re-labeling messages (e.g. re-labeling a 
message received at switch 2 on L12 / as the same message that is output from switch 2 but on 
L23 and which is received at switch 3, to be re-labeled by switch 3 and output again as L34) is 
known as "label bindmg ." Two or more LSRs, (network switch elements) which use a label 

1 0 distribution protocol to exchange label binding information are known as "label distribution 
peers" with respect to the binding information they exchange. 

The label distribution protocol also encompasses any negotiations in which two, label 
distribution peers, need to engage in order to leam of each other's MPLS capabilities. This 
label distribution protocol is referred to as path establishment signaling. MPLS defines two 

15 methods for label distribution. These two methods are: Label Distribution Protocol (LDP/CR- 
LDP) and ReSerVation Protocol (RSVP). 

Both LDP/CR-LDP and RSVP allow a path to be setup loosely (wherein each node 
determines it's next hop) or explicitly (wherein each node has been given it's next hop). 
These two protocols can be extended, as disclosed herein and by equivalents thereof, to 

20 provide a novel mechanism by which protection path establishment can be signaled and 
created. Accordingly, a "Protection" field can be defined, and added as an extension to the 
existing label request messages in LDP/CR-LDP, and path message in RSVP protocols. The 
destination or end point node in the MPLS network participates in setting up a recovery path 
as a merging network switch element. The destination or end point node learns, during a 

25 signaling or working/protection path configuration process, which working and protection 
paths are merged to the same outgoing network switch element. 

Hosts and routers that support both RSVP and Multi-Protocol Label Switching can 
associate labels with RSVP flows. When MPLS and RSVP are combined, the definition of a 
flow can be made more flexible. Once a label switched path (LSP) is established, the traffic 

30 through the path is defined by the label applied at the ingress node of the LSP (label switched 
path). The mapping of a label to traffic can be accomplished using a number of different 
criteria. The set of packets that are assigned the same label value by a specific node are said 
to belong to the same forwarding equivalence class (FEC) and effectively define the "RSVP 
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flow." When traffic is mapped onto a label-switched path in this way, we call the LSP an 
"LSP Tunnel". When labels are associated with traffic flows, it becomes possible for a router 
to identify the appropriate reservation state for a packet based on the packet's label value. 

A Path message travels firom a sender to receiver(s) along the same path(s) used by 
the data packets. The IP source address of a Path message must be an address of the sender it 
describes, while the destination address must be the DestAddress for the session. These 
addresses assure that the message will be correctly routed through a non-RSVP cloud. 

The format of an exemplary RSVP message with the Protection Object extension is: 

<Path Message> : := <Common Header> [ <INTEGRITY> ] 
<SESSION> <RSVP_HOP> 
[<riME_VALUES>] 
[ <EXPLICIT.ROUTE> ] 

[ <PROTECTION> ] /* The new message field. */ 
<LABEL_REQUEST> 
[ <SESSION_ATTRIBUTE> ] 
[ <POLICY_PATA> ... ] 
<sender descriptor> 

' Label Distribution Protocol (LDP) is defined for distribution of labels inside one 
MPLS domain. One of the most unportant services that may be offered using MPLS in 
general, and LDP in particular, is support for constraint-based routmg of traffic across the 
routed network. Constraint-based routing offers the opportunity to extend the information 
used to setup paths beyond what is available for the routing protocol. For instance, an LSP 
can be setup based on explicit route constraints, QoS constraints, and other constraints. 

Constraint-based routing (CR) is a mechanism used to meet Traffic Engineering. 
These requirements may be met by extending LDP for support of constraint-based routed 
label svsdtched paths (CR-LSPs). 

The Path Vector TLV is used with the Hop Count TLV in Label Request and Label 
Mapping messages to implement the optional LDP loop detection mechanism. Its use in the 
Label Request message records the path of LSRs the request has traversed. Its use in the 
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Label Mapping message records the path of LSRs a label advertisement has traversed to setup 
anLSP. 

The format of an exemplary CR-LDP message with the Protection TLV extension is: 

5 0 1 2 3 

01234567 8 9 0123456789012345678901 
+-+-+-+-+-+-+-4--+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-^^ 

|0| Label Request (0x0401) | Message Length | 
+.+.+.+.+.+_+_+.+_+_+.+.+-+.+.+.+.+.+.+.+.+.+_+.+-+^^ 

10 I Message© | 

I FECTLV I 
+.+.+.+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-^^ 
I LSPIDTLV (CR-LDP, mandatory) I 

15 +.+.+.+.+-+.+.+-+.+.+.+.+.+-+.+-+.+.+-+.+-+-+.+-+.+^^ 
1 ER-TLV (CR-LDP, optional) | 

+-+.+_+.+.+-+.+.+.+-+.+.+-+.+.+.+.+.+.+-+.+-+.+-+-+^ 

I Protection TLV (CR-LDP, optional) | 

+.+_+.+_+.+.+_+.+.+-+.+.+_+-+.+-+-+-+-+-+-+-+-+-+-+-^^ 

20 I Traffic TLV (CR-LDP, optional) | 

+.+-+.+.+.+.+.+.+.+-+.+-4--+-+-+-+.+-+-+-+.+-+-+.+-+^ 

I Pinning TLV (CR-LDP, optional) | 

+.+_+.+.+.+.+.+-+.+.+_+.+_+-+.+-+-+-+-+-+-+-+-+-+-+-+-^^^ 

I Resource Class TLV (CR-LDP, optional) | 
25 +.+.+.+.+-+-+-+-+-+.+.+.+.+.+-+.+-+.+-+-+.+-+.+-+-^^ 
I Pre-emption TLV (CR-LDP, optional) | 



30 



Wherein the 'Trotection TLV" message field is new. 

The Protection Object (RSVP)/Protection Type Length Value (TLV) (LDP/CR-LDP) 
establishes the working and a corresponding protection path utilizing the Reservation 
Protocol (RSVP) path message or the Constraint-Based Routing Label Distribution Protocol 
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(CR-LDP) Label Request message. The attributes required to establish the Protection Domain 
are: 

5 1 Priority: Specifies whether this protection group is a high or low switching priority. 

2 Protection Option: Specifies whether protection is supported. 

3 Protection Path Type: Specifies whether this establishment is for the Protection, or 
Working Path, 

4 Protected Path Identifier: Specifies a unique identifier for the protection traffic. 

10 5 Protection Node Type: Specifies whether the node is a switching, merging, or RNT root 
node. 

6 RNT Type: Specifies whether the RNT is created using Hop-by-hop, MPLS LSP, or 
SONET K1/K2. 

7 Timer Options: Specifies the hold off and notification time requirements. 

15 8 Recovery Option: Specifies whether the recovery action is Wait, Switch Back, or 
Switchover, 

9 Protection Bandwidth: Specifies whether the bandwidth of the protection path is available 
to carry excess (preemptable) traffic. 

20 

The following table illustrates the structure of an exemplary Protection 
Object/Protection TLV Structure. 

0 12 3 

25 01234567890123456789012345678901 

IPIDITI PGID |NID|RNTTl TO IROIB] RESVD \ 
+.4.-4-.+-+-+-+.+^+-+.+-+-+-+.+.+-+-+-+-+-+-+^^ 

30 P= Priority 

D= Protection Option 
T= Protection Path Type 
PGID= Protected path Identifier 
NID=Protection Node Type 
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RNTT=RNTType 
TO=Tmier Options 
RO=Recovery Option 
B= Protection Bandwidth 
5 RESVD-Reserved for Future Use 



Since the switching systems used in the network 1 00 are unidirectional, and pathway fault 
recovery requires the notification of faxxlts to a protection switch, such as switch no. 1 or switch 

10 no. 9, responsible for a switchover to a recovety path, a mechanism is provided for the fault 
indication and the fault recovery notification to travel firom a point of occurrence of the fault 
back to the protection switch. The abiUty to propagate a fault notice upstream however is 
compUcated when two or more data streams merge in a single switch such as the streams from 
switches 9 and 2 merging at switch 3. When two or more data streams merge at a switch, e.g. 

15 switch 9, a fault anywhere downstream from switch 9 will require that a fault notice be sent to 
multiple source switches, i.e. switches 9 and 2. The fault indication and recovery notification 
should be able to travel along a reverse path of the working paths to all the protection switch 
elements that might be affected by the fault. The path is provided by the reverse notification tree. 
By way of example, Figure 2 shows an example of an MPLS protection set up 

20 message sequence. The ^stabUshment of the working path and protection path is 

accomplished by the transmission of a Protection Switch Domain (PSD) initialization 
message 210 from a switch 202 to switches 204, 206 and 208. A PSD confirmation message 
212 is propagated from the downstream switch 208 upstream to switch 202. 

The reverse notification path, also referred to above as the Reverse Notification Tree 

25 or RNT, is established by the downstream switch, 208, sending an RNT initialization 
message 214, upstream to switches 206, 204 and 202. Confiirmation of the RNT setup is 
accomplished by the RNT Confirmation message 216 that originates from switch 202. 

Upon the establishment of the working and protection paths, and the reverse 
notification tree, data 218 can be sent through the network. 

30 Two "aliveness" messages 220 and 222, which provide notification of the working 

path status, are shown in Figure 2 to depict the fact that the aliveness message described 
above can be sent periodically, regardless of whether downstream data 218 was sent As 
shown fiirther, downstream data transmissions, such as transmissions 224, 226 and 228 are 
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not conditioned upon receipt of an aliveness message in any fixed way. An aliveness message 
230 sent upstream is then followed by yet another data transmission 232. 

Figure 2 shows that the sequence of aliveness messages and data transmissions do not 
need to follow any sort of predetermined or fixed order. For network reliability purposes, the 
5 aliveness messages are preferably sent periodically, so that their absence can be detected if 
they do not arrive on time. Altemate embodiments include sending liveness messages 
aperiodically. 

Those skilled in the art will recognize that re-routing data on either the failure of a 
link or a switch in a network such as that depicted in Figure 1 need not be performed by a 

10 protection switch. In the event that switch 4 fails for example switch no. 3 might reroute data 
fi-om switch 2 that is destined for switch 7, through another protection switch element 9, 
identified by reference numeral 116. Switch 9 might then reroute data fi*om switch 2 that is 
addressed to switch 7 over a protection path designed as Pg^io and Pio,? through switch 10, 
identified as reference numeral 118. 

15 In the preferred embodiment, the media over which data message are carried might be 

twisted copper Avires, coax cable, fiber optic cable or even a radio fi*equency data link. As set 
forth above, each of the switching systems might accommodate a variety of packetized data 
messages including but not limited to Ethernet, internet protocol, ATM, firame relay or other 
types of transmission switching systems. 

20 By continuously sending an upstream message indicating tiiat downstream traffic 

arrives at its destination, recovery time required to recover firom the fault of a media link or a 
switching system can be minimized. If the switch status message used to indicate a 
fimctionality of a switch or a link is sent promptiy enough, and to the appropriate node in a 
mesh network such as that shown in Figure 1, the time required to reroute data messages 

25 between first and second endpoint switches over an altemate data path can be minimized. In 
the preferred embodiment, the altemate or so called protection path is preferably set up in 
advance and maintained in a stand by mode such that it is inamediately available when 
required by the protection switch that will reroute data over the protection path. 
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We claim: 

1 . In a data network comprised of a plurality of data switches interconnected to form 
a plurality of data paths forming a mesh configuration of data switches, a method 
of re-routing data messages between first and second data switches over a pre- 

5 established alternate data path linking said fij:st and second data switches 

comprised of the steps of: 

a. sending at least a first data message over a first data path from said first switch 
to said second switch; 

b. receiving at said first data switch, switch status messages from said second 
10 switch; 

c. upon the loss of said switch status messages at said first switch, re-directing 
subsequent data messages over an alternate data path through said data 
network, 

2. The data network of claim 1 wherein said alternate data path is a protection path 
1 5 through said network. 

3. The method of claim 1 wherein said data switches are asynchronous transfer mode 
switches. 

4. The method of claim 1 wherein said data switches are internet protocol (IP) 
routers. 

20 5. The method of claim 1 wherein said switch status messages are comprised of a 
predetermined format, switch liveness message, 

6. The method of claim 1 wherein at least one of said switches maintains a table of 
incoming link and path identifiers and of outgoing link and path identifiers. 

7. The method of claim 1 wherein said first data message represents speech 
25 information. 

8. The method of claim 1 wherein said first data messages represents computer data. 
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9. The method of claim 1 wherein said step of re-directing said series of data 
messages from said first path over another path through said data network 
includes the steps of: 

sending subsequent first data messages to a third data switch. 
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10. The method of claim 1 wherem said first data switch is a protection switch 
element 

1 L In a data network comprised of a plurality of data switches interconnected to form 
a plurality of data paths forming a mesh configuration of data switches, a method 
of re-routing data messages around a data switch comprised of the steps of: • 

a. sending at least a first data message over a first data path fi:om a first switch to 
a second switch; 

b. sending said at least a first data message firom said second switch to a third 
switch; 

c. receiving at said second data switch, switch status messages indicating the 
fimctionality of said third data switch; 

d. upon the loss of said switch status messages at said second switch, sending a 
switch failure message firom said second switch to said first switch; 

e. upon the receipt of said switch failure message at said first switch, said first 
switch re-directing subsequent data messages away firom said second and third 
switch via a second data path through said data network. 

12. The data network of claim 1 1 wherein said second data path is a protection path 
through said network. 
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13. The method of claim 1 1 wherein said data switches are asynchronous transfer 
mode switches. 

14. The method of claim 1 1 wherein said data switches are intemet protocol (IP) 
routers. 

5 15. The method of claim 1 1 wherein said data switches are digital cross connect 
switches controlled by MPLS. 

16, The method of claim 1 1 wherein said data switches are optical cross connects and 
switches controlled by MPLS. 

17, The method of claim 1 1 wherein said switch status messages are comprised of a 
1 0 predetermined format, switch liveness message. 

1 8, The method of claim 1 1 wherein at least one of said switches maintains a table of 
incoming link and path identifiers and of outgoing link and path identifiers. 

1 9, The method of claim 1 1 wherein said series of data messages represent speech 
information. 

1 5 20. The method of claim 1 1 wherein said series of data messages represent computer 
data. 

2 1 . The method of claim 1 1 wherein said step of re-directing said series of data 
messages from said first path over another path through said data network 
includes the steps of: 

20 sending subsequent data messages to a third data switch. 

22. In a data network comprised of a plurality of data switches interconnected to form 
a plurality of data patihs forming a mesh configuration of data switches, a method 
of re-routing data messages between first and second data switches over a pre- 
established altemate data path linking said first and second data switches 

25 comprised of the steps of: 

24 



sending at least a first data message over a first data path fi-om said first switch 
to said second switch; 

upon the loss of said first data message at said second switch, sending a switch 
status messages to said first switch, the receipt of said switch status message 
thereby causing the re-directing of subsequent data messages over an altemate 
data path through said data network. 
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23. In a data network comprised of a plurality of data switches interconnected to fonn 
a plurality of data paths forming a mesh configuration of data switches, a method 
of re-routing data messages around a data switch comprised of the steps of: 

a. sending at least a fibrst data message over a first data path firom a first switch to 
a second switch; 

b. sending said at least a first data message firom said second switch to a third 
switch; 

upon the loss of said first data message at either said second switch or said 
third switch, sending a switch status message to at least one of said first and 
second switches thereby causing the re-directing of subsequent data messages 
away firom said second and third switch via another data path through said data 
network. 

24. The method of claim 23 wherein said first switch is a protection switch element. 
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